Forecast Evaluation for the German Forecast Hub
This is a preliminary evaluation of forecasts made for the German Forecast Hub. These evaluations are preliminary - this means I cannot currently rule out any mistakes and the plots and analyses are subject to change. The evaluations are not authorised by the German Forecast Hub team.
Feel free to reproduce this analysis. To that end, you can clone this repository and the Forecast Hub repository, specify the directory of the submissions files in the R-Markdown script and run the script.
If you have questions or want to give feedback, please create an issue in this repository.
Forecast Visualisation
Germany
2020-11-16
2020-11-09
2020-11-02
2020-10-26
2020-10-19
2020-10-12
Poland
2020-11-16
2020-11-09
2020-11-02
2020-10-26
2020-10-19
2020-10-12
Score Ranking
Overall
Overall separate
Germany
Poland
Score Overview
Overview of all the models and metrics
Overall
Overall separate
Germany
### Overall
2020-11-16
2020-11-09
2020-11-02
2020-10-26
2020-10-19
2020-10-12
Poland
### Overall
2020-11-16
2020-11-09
2020-11-02
2020-10-26
2020-10-19
2020-10-12
Overiew of Scores by Horizon
Overview of all models and metrics stratified by forecast horizons
horizon = 1
Overall
Overall separate
Germany
### Overall
Poland
### Overall
horizon = 2
Overall
Overall separate
Germany
### Overall
Poland
### Overall
horizon = 3
Overall
Overall separate
Germany
### Overall
Poland
### Overall
horizon = 4
Overall
Overall separate
Germany
### Overall
Poland
### Overall
Interval Score Components
Weighted interval score broken down into its components “overprediction penalty”, “underprediction penalty” and “sharpness”
Overall
Overall separate
Germany
### Overall
2020-11-16
2020-11-09
2020-11-02
2020-10-26
2020-10-19
2020-10-12
Poland
### Overall
2020-11-16
2020-11-09
2020-11-02
2020-10-26
2020-10-19
2020-10-12
Calibration
Interval Coverage
Overall
Overall separate
Germany
### Overall
2020-11-16
2020-11-09
2020-11-02
2020-10-26
2020-10-19
2020-10-12
Poland
### Overall
2020-11-16
2020-11-09
2020-11-02
2020-10-26
2020-10-19
2020-10-12
Plot of the percentage of true values captured by each prediction interval
Quantile Coverage
Overall
Overall separate
Germany
### Overall
2020-11-16
2020-11-09
2020-11-02
2020-10-26
2020-10-19
2020-10-12
Poland
### Overall
2020-11-16
2020-11-09
2020-11-02
2020-10-26
2020-10-19
2020-10-12
Plot of the percentage of true values below each predictive quantile level
Bias
Overall
Overall separate
Separate by date
2020-11-16
2020-11-09
2020-11-02
2020-10-26
2020-10-19
2020-10-12
